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IMiversities increasin^y adopt 
innovative teaching models, 
which focus on the devel- 
opment of skills instead of 
the reproduction of knowl- 
edge. These new teaching 
models emphasise the im- 
portance of knowledge ap- 
plication and the develop- 
ment of competencies. Yet, 
using these teaching meth- 
ods usually impUes a hi^ as- 
sessment burden for lectur- 
ers. Educational institutions 
are on the look-out for ways 
of offering competency-ori- 
ented teaching materials to 
students and keeping the as- 
sessment burden for lectur- 
ers down at the same time. 
This article gives a practical 
illustration of a marketing 
course developed by the Fac- 
ulty of Management Sciences 
of the Open University of The 
Netherlands. This course 
shows a successful way in 
which both aims can be 
achieved simultaneously. 


(') For an histniical overview of dis- 
tance education see Passerini and 
Grander, 2000 and Spooner et al., 

1998. 

(0 NIMA is the Nederlands Institu- 
ut voor Marketing (Institute for Mar- 
keting of the Netherlands). The NDMA- 
A examination assesses whether stu- 
dents possess the knowledge and 
the skills to operate on an execu- 
tive level within the marketing dis- 
cipline. The NIMA-A examination is 
oriented towards general marketing 
concepts and principles, marketing 
instruments and marketing envi- 
ronments and assesses whether stu- 
dents are able to actively apply their 
knowledge and skills in practice. 
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Teaching competencies 
efficientiy through the 
Internet - a practical 
example 


Introduction 

Can students leam competencies through 
an Internet course? Do guidance and as- 
sessment of a competency-oriented course 
not necessitate far too much supervisoiy time 
on the part of teachers? These are questions 
with which educational institutions in dis- 
tance teaching wrestle. 

Educational institutions are continuously un- 
der pressure to operate cost-effectively (Moo- 
nen, 1994). It is therefore important to keep 
down the time teachers spend on marking 
examinations. But is this still possible now 
that education is shifting from being knowl- 
edge-oriented to competency-oriented? 
Examinations using multiple-choice ques- 
tions have the advantage that they can be 
easily checked automatically. As a result, 
they are cost-effective. However, multiple- 
choice examinations are specifically designed 
for assessing knowledge-oriented courses 
(Van der Vleuten, 1996). It is no longer pos- 
sible to make do with multiple-choice ques- 
tions in examining competencies (Daniels 
and van der Vleuten, 2002). Open questions 
and project assignments are often the most 
appropriate way of examining competen- 
cies. Yet, using these methods usually im- 
plies a high assessment burden for teachers, 
since they have to assess every single as- 
signment manually. 

As a result, educational institutions are on 
the lookout for ways of offering competen- 
cy-oriented teaching materials to students 
and keeping the assessment burden for teach- 
ers down at the same time. An unrealisable 
task as it seems. In this article we will give 
an example of how it is possible to ac- 
complish both aims simultaneously. We will 
use a marketing course as an illustration. 
This course has been developed by the Fac- 
ulty of Management Sciences of the Open 
University of the Netherlands. First, we will 


discuss the background of the students and 
we will briefly review the programme that 
features the marketing course. After this, we 
will describe the competency-oriented na- 
ture of the course and we will explain how 
the exact set up of the course permits little 
guidance and assessment time. Finally, we 
will overview results from a survey held 
among students that finished the course. 

Background of the students and 
the course 

The Open University of the Netherlands is 
an educational institute that is specialised in 
distance education. A typical student of the 
Faculty of Management Sciences is in his/her 
thirties, has a job in management and wants 
to deepen his/her knowledge in areas such 
as marketing, accounting and organisation- 
al behaviour and culture. The Management 
Sciences programme consists of courses in 
all these fields and more, including eco- 
nomics, human resource management and 
public administration. All courses are offered 
by distance education. (‘) This entails that 
study materials are designed for self-study 
and students receive study materials at home 
(by regular and/or electronic mail and/or 
through the Internet). The communication 
between teachers and students takes place 
through e-mail, postings in discussion groups 
and telephone calls. Exams are generally tak- 
en in a classroom. Bachelor degree cours- 
es often use multiple-choice examinations, 
whereas master degree courses often involve 
writing assignments and project reports. 

The Introduction into Marketing is a course 
at NIMA-A-level (^) offered in the bachelor 
phase to students of the Open University of 
the Netherlands. This innovative course start- 
ed in September 2002. The course covers 
several fields, (see Table 1 for an overview 
of the course topics). Students are consid- 
ered to spend 200 hours of study time on 




this course. The written and electnonic course 
materials can be studied independently, at 
the student's own pace and without the 
assistance of a teacher. If there is never- 
theless an occasional need for substantive 
guidance, the student can contact a super- 
visory teacher (by telephone or e-mail). 

Competency -oriented learning 

Universities increasingly adopt innovative 
teaching models, which focus on the de- 
velopment of skills instead of the repro- 
duction of knowledge. These new teaching 
models emphasise the importance of knowl- 
edge application and the development of 
competencies (Enkenberg, 2001; Parikh and 
Verna, 2002). In traditional education, stu- 
dents study the textbook and apply their 
newly acquired knowledge to assignments. 
In a competency-oriented course students 
begin with assignments which invite them 
to solve practical problems. The Introduc- 
tion into Marketing is a good example of a 
course that adopts competency-oriented 
teaching. The principle underlying the course 
is to develop in students the skill to apply 
theory to practical cases. In addition, stu- 
dents develop the ability to actively search 
for relevant information both from theory 
(textbook) and from practice (using case 
studies). After completing this course, stu- 
dents are able to work in a result-oriented 
manner and they can sensibly adapt and ap- 
ply methods and techniques from market- 
ing to practical situations. 

The general design of the course is as fol- 
lows. The student is guided by an electron- 
ic study manual that contains assignments 
and short case studies on sixteen different 
marketing topics (see Table 1). The assign- 
ments and cases are offered through the In- 
ternet. These are interactive and make use 
of multimedia. A separate textbook is sent 
to the students' homes. The book can be 
used to find relevant theoretical information 
while solving the assignments. It provides 
in-depth theoretical knowledge on each of 
the sixteen topics of the course. 

For each of the sixteen marketing topics the 
electronic study manual offers three com- 
ponents through the Internet These are: 

□ a self-test 

□ exercise assignments in an exercise case 
study 


Overview of the course topics Table 1 

Part 1 - Orientation and introduction/ overview 

1. Overview of marketing 

2. Strategy development 

3. The marketing environment 


Part 2 - Analysis of the market 

4. Consumer behaviour 

5. Marketing research and marketing information 

6. Markets and segmentation 


Part 3 - Product marketing 

7. Product positioning 

8. New product development 


Part 4 - Advertising & promotion 

9. Marketing communication strategies 

10. Advertising and branding 

11. Sales management, direct marketing and promotion 


Part 5 - Pricing 

12. Price policy 

13. Factors affecting price decisions 


Part 6 - Distribution 

14. Distribution policy 

15. Marketing channels 


Part 7 - Marketing planning 

16. Organisation, planning en control) 


□ examination assignments in an exami- 
nation case study. 

Self-test 

Students can test the level of their own knowl- 
edge with ten multiple-choice questions. 

After each question, students automatically 
receive feedback on the answer they have 
filled in. It is clearly indicated in the feed- 
back what is right or wrong in the answer 
chosen. Empirical evidence indicates that 
simply showing students the correct answer 
has less effect on learning than providing 
an elaborate feedback on the correct and in- 
correct elements of the given answer 
(Dempsey, Driscoll and Swindell, 1993). 

Moreover, the motivation of students is pos- 
itively influenced when feedback is given 
that is tailor made to the answer of the 
student (Ross and Morrison, 1993). 
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In addition, the feedback shows which parts 
of the textbook provide further in-depth 
study material on the topic of the multi- 
ple-choice question. On the basis of the er- 
rors made in ten questions, students ob- 
tain an overview of the knowledge they lack. 
This gap in knowledge must be filled (by 
studying the textbook) before a start can be 
made on practising competencies for this 
specific marketing topic. 

Exercise assignments 

In a competency-based approach, assign- 
ments provide the guidance for the learn- 
ing process of students. In this course, stu- 
dents have to complete a number of exer- 
cise assignments on each topic in the elec- 
tronic study manual on the Internet These 
exercise assignments relate to an exercise 
case study. An exercise assignment is ap- 
phcation-oriented, for example 'Describe the 
marketing mix for Center Parcs' or 'Assess 
the communication strategy of Peugeot'. The 
idea of using application-oriented assign- 
ments forms the core of the teaching meth- 
ods in this course. The assignments (and not 
the case study) are the point of departure 
for the student. To complete an assignment, 
the student first needs theoretical knowl- 
edge (what does the concept of 'market- 
ing mix' entail - or - how does one assess 
the communication strategy of a company)). 
Second, the student needs practical knowl- 
edge of the case with which the assignment 
is concerned (how does Center Parcs fit to- 
gether? or what is there to know of Peugeot). 
A hyperlink from a theoretical term in the 
assignment to a glossary (see Figure 1) guides 
the students to the theoretical knowledge 
they need. As well as a brief explanation of 
the term, the glossary also contains a ref- 
erence to a section in the textbook where 
this specific term is explained in more de- 
tail. Another hyperlink from the assignment 
refers to the case-study materials (see Fig- 
ure 1). The case-study materials provide in- 
formation resources on a specific compa- 
ny or practical situation. 

Although the student has access to all the 
case-study information, he/ she still has to 
decide for him-/herself what information 
is needed in order to answer the assign- 
ments. In this way, the student learns how 
to distinguish relevant from irrelevant in- 
formation. The case-study materials include 
various links to relevant sites of compa- 
nies and institutions. In addition, the text is 


enriched with pictures, advertising film clips 
and on-line newspaper articles. This makes 
the case study alive for the student The stu- 
dent feels that he/ she is in a motivating learn- 
ing environment which invites him/her to 
think up solutions to the assignments. Ap- 
praisals among students show that this is al- 
so found to be the case (see the survey re- 
sults further on in this article). 

The approach as described above means that 
the students' work is very result-oriented. 
All the student's work is directly geared to- 
wards answering the assignment. A student 
does not study the textbook until the mo- 
ment when he/ she needs the specific the- 
oretical knowledge necessary to complete 
the assignment. The theoretical knowledge 
drawn from the textbook can be immedi- 
ately applied to the practical assignment. 
Therefore, the course contains key aspects 
of just-in-time learning (Schoening, 1998). 

After the necessary knowledge has been ac- 
quired firam theory and from the case, the 
student can type in the answer on-line in an 
answer window (see Figure 1). As soon as 
an answer is filled in and saved, a supervi- 
sory teacher can view the answer given. The 
answer has to be typed in and not just vague- 
ly thought up, which prompts the student 
to express him-/herself well. The possibili- 
ty of a supervisory teacher being able to see 
the student's answer has also the effect that 
the student will take care in filhng in the an- 
swer. Students will do their best to justify 
the answers properly. 

After the answer has been filled in and saved, 
the student can ask for feedback. The stu- 
dent is then shown a standard answer to the 
assignment. This answer often focuses on 
common errors. In addition, references to 
relevant passages from the textbook are in- 
cluded for interpretation of the marketing 
terms used. The student now learns from 
the differences between his/her own reply 
and the standard reply and can fill any re- 
maining gaps in his/her knowledge by con- 
sulting the textbook. 

Examination assignments 

Finally a number of examination assign- 
ments in an examination case study are pre- 
sented to the student for each of the sixteen 
marketing topics. This is again done com- 
pletely digitally. These assignments are of 
the same level as the exercise assignments 





in the exercise case study. A similar proce- 
dure is followed to that described above for 
the exercise assignments and the exercise 
case study. The difference compared with 
the exercise assignments is that no feedback 
is given for examination assignments. This 
is justified by the underlying notion that 
learning takes place during the completion 
of the exercise assignments, and the ex- 
amination assignments are purely geared 
towards testing. 

Examination and assessment 

The design of the examination procedure is 
based on four requirements: 

□ A low assessment burden must be at- 
tained, while competencies are nevertheless 
tested. 

□ Students must be able to be examined 
(and therefore assessed) at any chosen mo- 
ment. 

□ The assessment must be objective, and 
therefore independent of the teacher who 
carries out the assessment 

□ Cribbing of answers must be visible 
straightaway, so that action can be taken. 

The solution that has been designed deals 
with each of the above issues. 

Testing competencies with a low as- 
sessment burden 

It is a general rule that what students have 
been taught must be examined (Brown, 
1996). It is even the case that the method of 
testing determines what is learned (Prodro- 
mou, 1995; Gipps and James, 1996). In the 
Introduction into Marketing, students are 
taught to apply theoretical knowledge to 
practical case studies through the exercise 
assignments and exercise case studies. It is 
not possible to test whether students have 
acquired this skill during the course by us- 
ing multiple-choice questions. Consequent- 
ly, we use examination assignments in ex- 
amination case studies for the assessment of 
students. In this way, we examine exactly 
the competency that has been taught. 

How does this principle tally with keeping 
down the assessment burden for teachers? 
Does this way of working not lead to teach- 
ers spending a large amount of time on maik- 
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ing student answers? An average examina- 
tion case study consists of six assignments. 
The course includes sixteen marketing top- 
ics, each of which contains one examina- 
tion case study. This means that each stu- 
dent completes over 100 examination as- 
signments. We have nevertheless succeed- 
ed in keeping down the assessment burden 
for teachers. The following approach has 
been adopted. 

Of the sixteen examination case studies that 
the student completes, four are actually cor- 
rected. The student is allowed to choose two 
out of these four case studies to be assessed, 
while the other two case studies are drawn 
at random from the other fourteen exami- 
nation case studies the student has made. 
The advantage of this method of examina- 
tion is that the student will take care with 
aU examination assignments, as he/she does 
not know beforehand which two case stud- 
ies will be chosen at random by the elec- 
tronic system to be added to the set of case 
studies actually assessed. As students choose 
two case studies themselves, they have some 
influence overtire set of case studies looked 
at. This prevents the argument afterwards 
(in the case of a failure) that it is the worst 
case studies that have been included in 
the assessment Broadly speaking, this method 
of examination does not have any draw- 
backs for students, while the approach of- 
fers great benefits to the teachers. Only four 
case studies have to be assessed, not six- 
teen. Teacher assessment time is therefore 
considerably reduced. 

Other features of the electronic system that 
make assessment easier for teachers are dis- 
cussed in the remainder of this article. 

Continuous examination and continu- 
ous assessment 

It has been ensured in the Introduction 
into Marketing that the teacher is able to as- 
sess on-line through the electronic system. 
After logging on, the teacher is presented 
with a list of students to be assessed by 
him/her. The teacher chooses a particular 
student and views all the assignments with 
the student's answers (Figure 2, left). The 
electronic system indicates the maximum 
number of marks to be allocated to this 
assignment. The teacher compares the stu- 
dent's answer with the answer model. All 
the elements that a good answer must con- 
tain appear in the answer model. The teacher 
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On-line assessment by teachers 


Figure 2 




Note: students' answers and teachers' comments are fictional. 
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can then indicate on-line how many marks 
he/ she is willing to award to the answer. 
Until assessment of the student is finished, 
the assessment of each assignment can still 
be adjusted. When the teacher has finished 
assessing a student, he/ she clicks on a par- 
ticular button and the assessment has then 
been finally completed. 

Electronic examination ensures that no pa- 
per routes are necessary. Students do not 
need to send their written examinations to 
a central point. Nor do the examinations 
submitted need to be manually distributed 
and sent to assessing teachers. This signifies 
a gain in teacher productivity and an im- 
provement in efficiency. 

This way of working makes it possible to 
assess examinations throughout the year 
with a group of three teachers. Students can 
submit their examinations at any time they 
want Teachers know their individual review 
periods in which they are responsible for 
correcting the assignments that are submit- 
ted electronically by the students. 

How to achieve objectivity in assessment 

The Introduction into Marketing is assessed 
by a team of three teachers. How is it en- 
sured that the assessment of a student is in- 
dependent of the teacher who carries out 
the assessment? In the electronic system ob- 
jectivity is guaranteed by what is known 
as the answer history. The teacher can eas- 
ily see how other teachers have assessed 
students' answers on a particular question 
(Figure 2, right). This makes it possible to 
achieve unity in assessment between dif- 
ferent teachers. In addition, the answer his- 
tory makes it easier for teachers to allocate 
a score to an answer. A teacher benefits finm 
the way other teachers have already weighed 
up this assignment The answer history re- 
veals statements such as: '1 deduct one mark 
for lack of good argumentation'. 

What to do about cribbing 

There is a fear in electronic examination that 
students will exchange answers among them- 
selves. Cribbing quickly becomes visible 
to teachers as a result of using the answer 
history. With the answer history, the teacher 
obtains a list of all answers students have 
submitted on a particular assignment. It is 
immediately clear whether two answers are 
exactly the same. 
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Nor does cribbing guarantee a pass for the 
student This is due to the specific design of 
the examination procedure in which two case 
studies are drawn at random by the elec- 
tronic system. The student can only partly 
decide for him-/herself which case studies 
wiU be assessed. In addition, a total of more 
than 100 examination assignments are sub- 
mitted (for sixteen different examination case 
studies) before a student can be assessed. 
This discourages students from enlisting some- 
one else to take the examination. 

We also wish to prevent successful students 
from circulating 'correct' answers. Students 
who have already passed therefore only re- 
ceive limited feedback on errors they have 
made in their answers. The exercise envi- 
ronment provides feedback on errors made, 
while the examination environment is sole- 
ly intended to test how well the student mas- 
ters the acquired skills. Errors made in the 
examination are therefore only explained 
verbally. Note that students of the Open 
University take part individually in courses. 
They start the course at different dates and 
they have different study speeds. Moreover, 
students do not know one another, which 
limits the opportunity to exchange 'cor- 
rect' answers. 

Survey results 

The course has been evaluated among stu- 
dents. Five students were asked to give their 
opinion on various aspects during the course. 
These students have filled in two ques- 
tionnaires. One questionnaire handled is- 
sues such as study load and difficulty, de- 
sign of the course, usefulness and ease of 
use of the electronic manual. Furthermore, 
questions were asked on whether the elec- 
tronic manual was convenient and stimu- 
lated study. This questionnaire was com- 
pleted at the end of the course. The second 
questionnaire was filled in during the course 
and had open questions on whether the stu- 
dents encountered problems with specific 
assignments. This could be a technical prob- 
lem but also a content-related problem. The 
general opinion of these five students was 
that the electronic study manual was easy 
to use and motivated further study of the 
textbook and related study materials. Hard- 
ly any technical problems were encountered. 
Students unanimously reported to be very 
pleased to work on assignments related to 
practical situations. 



\fiP])03NI'(QW 


In addition, we have administered an eval- 
uative survey among 32 students that fin- 
ished the course. Seventeen students com- 
pleted the questionnaire (response rate of 
53.1 %). Again, questions were directed at 
the usefulness and the ease of use of the 
electronic manual as well as the study load 
and difficulty of the course. 

The questionnaire contained mostly open 
questions. On the question 'To what degree 
do you think that the use of the electronic 
manual offers added value above the use of 
a paper manual" the majority of the students 
(10 out of 17) was very positive, stating that 
the electronic manual was very efficient and 
easy to use. One student explicitly stated that 
the electronic environment stimulated the di- 
gestion of the study materials and encour- 
aged the practical engagement in the taught 
discipline. Two students did not have a pref- 
erence for either an electronic or a paper 
study manual and five students did not see 
any additional advantages of an electronic 
manual compared to a paper manual. 

On the question whether the specific design 
of the examination suited the course, five 
students disagreed, while twelve students 
were very positive. One student stated, 'This 
method gives students the possibility to 
choose two cases that are closely related to 
their own interests". One of the disagreeing 
students thought that it would be better if 
aU four cases were chosen at random by the 
electronic system. He/ she was afraid that 
students would focus on the two cases of 
their choice and only have a half-hearted at- 
tempt on the other cases. All students re- 
ported to be pleased to work on assignments 
related to real-life situations. Table 2 shows 
several statements of students regarding this 
issue. Students reported an average appraisal 
of 7,9 on a 10-point scale (10 being the max- 
imum score) satisfaction level for tire course. 
Students stated that the Introduction into 
Marketing is "an entertaining course, an ex- 
cellent way to test competencies". 

Besides the evaluation among students, two 
teachers were interviewed for their experi- 
ences with the assessment of the course. 
The teachers stated that the on-line as- 
sessment was easy to use. The electronic 
assessment environment gives the teacher 
a view on both the student's answer and 
the answer model. Teachers set great value 
to the fact that they do not have to search 
their papers for the answer model and the 


Students' statements Table 2 


- "Very good course that encourages you to a structured and analytic approach of 
practical situations" 

- "You can better understand the marketing concepts and theories by applying 
them to practical situations" 

- 'This course forces you to think!!!" 

- 'The material encourages you and makes you crave for more" 

- 'The course makes marketing alive" 

- 'Tou learn a lot by analysing the cases" 

- "It is enjoyable to alternately read and make practical assignments" 


opinion of other teachers on similar answers. 

They stated that the electronic system made 
the grading process accurate and very pleas- 
ant to do. Furthermore, the teachers stat- 
ed that the system made it possible to grade 
a student's performance within half an hour. 

In comparison, at the Faculty of Manage- 
ment Sciences, an average competency-ori- 
ented course has an assessment burden of 
one hour per student. The specific design 
of the Introduction into Marketing course 
therefore reduces the regular assessment 
burden of a course by fifty percent 

Summary and concluding remarks 

At first sight competency-oriented educa- 
tion does not appear to go together with 
cost-effectiveness. It involves an assessment 
with open questions, which usually puts a 
large burden on teachers. This article has 
given an example of a course that achieves 
competency-oriented education with a rel- 
atively small assessment burden. First the 
Introduction into Marketing course is geared 
towards the development of competencies 
among students by giving them assignments 
that pertain to practical cases. Moreover, the 
extent to which the student has acquired 
these competencies is examined in the course. 

Therefore, the examination is in close keep- 
ing with the way in which marketing has 
been taught Second, effective use has been 
made of all the benefits that the Internet of- 
fers to keep down the assessment burden 
on teachers. As a result teacher productiv- 
ity is raised and teaching costs are kept down. 

Evaluations of the course showed that stu- 
dents as well as teachers are very satisfied 
with the electronic system. This course could 
provide the basis for other courses that aim 
to achieve cost-effective competency-ori- 
ented learning materials. Innovative solu- 
tions like this course make it possible to 
meet the challenges of current education. 
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